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PREFACE 

An  adaptive  controller  is  one  which  has  the  capability  to  learn 
from  experience  and  in  so  doing  improves  the  quality  of  the  control 
it  exerts  over  a  plant.  Such  a  controller  adapts  itself  to  circum¬ 
stances  as  it  finds  them.  The  potential  military  and  commercial 
applications  of  such  devices  are  impressive. 

The  aim  of  this  Memorandum  is  to  show  that  the  basic  system 
identification  and  state  determination  problems  can  be  viewed, 
mathematically,  as  nonlinear  multipoint  boundary  value  problems.  They 
can  be  resolved,  computationally,  via  use  of  quasilinearization  and 
differential  approximation.  The  net  result  is  that  we  now  possess 
straightforward  and  effective  procedures  for  designing  vide  classes  of 
adaptive  controllers.  This  report  may  be  of  interest  to  control 
engineers  and  numerical  analysts. 


A. 


SUMMAKI 


Suppose  that  a  system  undergoes  a  process  described  by  a 
system  of  differential  equations.  The  equations  contain  some  un¬ 
known  parameters  and  not  all  the  initial  conditions  are  known. 
Observations  are  made  on  sane  of  the  state  variables  during  the 
course  of  the  process.  We  wish  to  determine  the  system  parameters 
and  initial  conditions  which  lead  to  best  agreement  with  the 
observations . 

Inverse  problems  of  the  type  sketched,  which  are  important 
throughout  mathematical  physics  and  engineering,  tire  cast  in  mathe¬ 
matical  form  as  nonlinear  multipoint  boundary  value  problems.  Then 
computational  solutions  via  quasilinearization  and  differential 
approximation  are  suggested. 
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I.  INTRODUCTION 

What  distinguishes  an  adaptive  control  system  from  conventional 
control  systems  has  been  a  matter  of  great  discussion  and  controversy. 
It  has  even  been  pointed  out  that  any  control  system  is  adaptive  if 
one  chooses  to  define  certain  functions  of  a  control  system  in  a 
suitable  manner. 

A  commonly  accepted  definition  of  an  adaptive  system  is  the  fol¬ 
lowing:  An  adaptive  control  system  is  one  in  which  certain  parameters 
of  the  system  are  adjusted  to  counteract  degradations  in  performance 
brought  about  by  changes  in  the  system's  environments.  Even  this  de¬ 
finition  is  rather  vague.  With  this  definition  the  same  control  system 
can  be  adaptive  or  non-adaptive,  depending  on  its  block  diagram  re¬ 
presentation.  For  examples  of  this  situation  see  Ref.  2.  In  this 
Memorandum  the  operational  definitions  given  in  the  reference  will  be 
used  to  distinguish  adaptive  from  non-adaptive  control  systems .  These 
definitions  will  be  outlined  in  a  later  section.  The  plant  refers  to 
the  object  to  be  controlled,  and  system  refers  to  the  combination  of 
the  plant  and  controller. 

A  control  problem  which  can  often  be  solved  by  conventional  tech¬ 
niques  is  the  following:  Given  a  plant  whose  characteristics  cannot 
be  changed  by  the  designer,  and  given  some  performance  specifications 
and  constraints,  find  a  satisfactory  controller  to  actuate  the  plant. 

If  the  input  to  the  controller  is  sensitive  only  to  the  system  input, 
the  resulting  system  is  called  open  loop.  This  will  yield  a  satisfac¬ 
tory  system  only  if  the  plant  and  its  environment  and  the  expected 
inputs  sure  precisely  known  beforehand.  In  general,  this  will  not  be 
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the  case.  When  open  loop  control  is  not  possible,  the  controller 
should  also  operate  on  the  plant  output  and  the  system  is  then  called 
closed  loop.  For  a  closed  loop  system  design  to  be  satisfactory, 
the  plant  character! sties  and  their  relationship  to  environmental 

changes  should  be  known  beforehand. 

Closed  loop  controller  design  may  be  viewed  in  the  following 
manner.  Before  starting  the  design  procedure,  the  plant  characteris¬ 
tics  are  known,  i.e.,  the  plant  has  been  identified.  Based  on  certain 
performance  specifications  or  performance  indices,  a  controller  is 
designed,  i.e.,  a  decision  is  made  concerning  the  type  of  controller 
to  be  used.  Finally,  based  on  this  decision,  certain  signals  or 
parameters  of  the  system  must  undergo  modification  to  accomplish 
this  result. 

Now  consider  the  problem  of  designing  a  controller  for  a  plant 
with  incompletely  defined  dynamical  characteristics.  If  the  dynami¬ 
cal  characteristics  do  not  change  very  rapidly,  one  can  still  view 
this  problem  as  a  closed  loop  controller  design.  The  only  difference 
is  that  the  controller  will  be  different  for  different  environmental 
conditions.  In  such  cases  it  may  be  advisable  to  so  organize  the 
over-all  system  that  it  performs  certain  of  the  functions  conven¬ 
tionally  exercised  by  the  designer  himself  during  the  design  phase. 
That  is ,  the  controller  may  be  called  on  to  compute  or  identify  the 
characteristics  of  the  plant  while  the  system  is  in  normal  operation. 
The  controller  must  then  make  a  decision  concerning  the  way  in  which 
the  available  parameters  of  the  system  should  be  adjusted  so  as  to 
improve  the  operation  with  respect  to  a  defined  index  of  performance. 
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Finally,  certain  signals  or  parameters  must  undergo  modification  to 
accomplish  this  result. 

Consistent  vith  the  definition  in  Ref.  2,  a  control  system  which 
accomplishes  these  three  functions  of  identification,  decision,  and 
modification  will  he  termed  an  adaptive  control  system. 

A  schematic  representation  of  an  adaptive  control  system  is  shown 
in  Fig.  1. 

Environment 


Fig.1  —  Schematic  of  an  adaptive  control  system 


In  Fig.  1,  double  lines  indicate  vector  valued  signals.  The 
identification  function  is  performed  by  the  identifier  and  the  deci¬ 
sion  and  modification  functions  are  performed  by  the  optimal  control¬ 
ler.  The  combination  of  the  identifier  and  optimal  controller  will 
be  called  an  adaptive  controller. 
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In  this  Memorandum  the  identification  and  decision  problems  vill  he 
separately  considered,  and  methods  for  obtaining  their  solutions 
proposed.  One  method  of  solution  depends  on  converting  the  identifi¬ 
cation  and  decision  problems  to  a  problem  of  solving  nonlinear  or¬ 
dinary  differential  equations  with  multiple  point  boundary  values 
specified  and  solving  the  latter  problem  by  the  method  of  quasi- 
linearizationl^  Another  method  depends  on  using  differential  ap- 

(4) 

proximation  techniques . 

For  an  excellent  survey  of  other  methods  of  solving  certain 
classes  of  adaptive  control  problems  and  the  historical  development 
of  this  area,  refer  to  Gibson. See  also  Ref.  6. 


■ 
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II.  THE  IDENTIFICATION  PROBLEM 

Jfost  of  the  identification  schemes  proposed  in  the  control 
engineering  literature  involve  determination  of  the  impulse  response 
of  plants  by  either  subjecting  them  to  specific  test  signal  inputs 
or  using  normal  operating  data.  These  schemes  suffer  from  the  dis¬ 
advantage  that  they  are  useful  only  for  linear  time- invariant  plants. 

The  identification  schemes  proposed  here  are  equally  applicable 
for  linear  and  nonlinear  plants.  It  is  assumed  that  the  plants  are 
described  by  ordinary  differential  equations  whose  forms  are  known 
except  for  a  finite  set  of  unknown  parameters  which  depend  on  the 
environments.  The  parameters  are  assumed  to  be  constant  over  the 
time  required  to  identify. 

The  assumption  regarding  knowledge  of  the  form  of  the  differen¬ 
tial  equation  is  not  restrictive  in  practice  since  this  can  either 
be  obtained  from  physical  considerations  or  by  letting  the  identifier 
go  through  an  initial  period  of  "learning"  over  which  the  best  form 
that  relates  the  input-output  data  may  be  determined. 

Let  the  dynamical  equations  of  the  plant  be  of  the  form 

x  =  f(x,u,a)  (1) 

where  x  is  an  n-dimensional  vector  (x^, ...,xn),  the  state  of  the 
system;  u  is  a  scalar,  the  control  function;  f  is  an  n-dimensional 
vector  (f^, ...,fn);  and  a  is  a  k-dimensional  vector  representing  the 
k  parameters  of  the  plant  which  depend  on  environmental  conditions. 

More  generally,  u  can  be  an  m-dimens ional  vector  (i.e.,  the 
plant  is  a  multi-variable  plant)  and  f  can  be  an  explicit  function 
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of  time  t  (i.e.,  the  plant  is  a  time  variable  plant).  The  generaliza¬ 
tion  of  the  methods  proposed  below  for  these  cases  is  straightforward 
and  hence  only  the  special  case  of  plants  represented  by  Eq.  (1)  will 
be  considered  here. 

The  two  schemes  for  solving  the  identification  problem  are  the 
following . 

(7) 

A.  THE  QUASILINEARIZATION  METHOD  FOR  IDENTIFICATION^  ' 

To  use  this  method,  the  identifier  should  have  the  following 
information.  The  input  to  the  plant,  u(t),  should  be  known  in  the 
interval  0  £  t  £  T,  where  T  is  a  suitably  chosen  constant.  In  addi¬ 
tion,  observations  of  the  output  at  certain  inBtants  of  time  should 
also  be  known.  The  observations  need  only  be  on  the  physically 
available  components  of  the  state  x  of  the  plant.  let  y  be  a  subset 
of  x  where  the  components  of  y  are  physically  available  for  measure¬ 
ment.  Thus,  the  known  observations  are 

y(t±)  =  cjL  (2) 

i  =  1,2,...  ,m  with  t  s  T  and  m  is  a  suitably  chosen  number  such  that 
Eq.  (2)  yields  (n-tk)  conditions. 

The  parameter  vector  a  is  really  a  constant  in  the  interval 
0  s  t  £  T.  However,  we  shall  treat  it  as  a  function  of  time  and 
adjoin  to  the  plant  Eq.  (l),  the  equation 

a  =  0  (3) 

The  question  now  is  the  following.  What  is  the  solution  to  the 
(n+k)  first  order  Eqs.  (l)  and  (3)  subject  to  the  (n+k)  boundary 
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conditions  represented  by  Eq.  (2)?  This  is  a  multipoint  boundary 
value  problem  to  be  solved  computationally  using  the  method  of  quasi¬ 
linearization  outlined  in  Refs.  3>  1,  8,  9>  10. 

s  t 

The  linearized  equations  for  the  (n+l)  approximation  are 


'(n+l)  _ 


=  f(S<n>,u,a(n>) 


+  f 


(n) 


(5 


(n+l) 


(*) 


a<n+l)  =  0 


with  boundary  conditions 


(n+l)  *  _ 

y  (t±)  =  c± 


(5) 


In  Eq.  (4) 


,(n) 


*1 


df 
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*1 


n 


bf 

n 

ST 
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x 
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=  x 


=  a 


(n) 

(n) 


(6) 


,(n) 


is  the  value  of  x  in  the  n 


th 


and  f  ^  is  similarly  defined,  x  _ 

a  /Q\ 

iteration  and  ar  ’  has  a  similar  meaning. 

The  solution  of  Eqs.  (4)  and  (5)  yields  the  value  of  the  parameter 
vector  a.  Notice  that  in  addition  to  determining  the  values  of  the 
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unknown  parameters  the  inaccessible  states  will  also  "be  automatically 
determined  by  this  procedure*  which  is  quadratics! 1 y  convergent. 

B.  THE  DIFFf^IfflAL- APPROXIMATION  METHOD  FOR  IDENTIFICATION 

The  current  estimate  of  the  plant  parameter  vector  a  is  the 
one  that  makes,  from  Eq.  (1) 

x  -  f (x,u,a)  s  o  (7) 

the  identity  holding  over  the  identification  time.  Any  other  value 
of  the  parameter  vector  will  not  satisfy  Eq.  (7)*  This  simple  ob¬ 
servation  rnnh-PB  it  evident  that  the  true  value  of  the  plant  parameters 
is  the  solution  of 


J. 

J  <  x  -  f(x,u,a)  ,  x  -  f (x,u,a)  > 


where  T  is  a  suitably  chosen  quantity.  In  Eq.  (8)  <  •  >  denotes  the 
Euclidean  inner  product. 

This  is  a  straightforward  minimization  problem  which  yields  upon 
taking  partial  derivatives  of  the  integral  with  respect  to  the  com¬ 
ponents  of  a  and  setting  them  equal  to  zero 


In  Eq.  (9) 


f '  = 
a 


^ 
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Equation  (9)  represents  k  simultaneous  equations  whose  solution  yields 
the  plant  parameter  vector  a. 

This  particular  method  of  identification  appears  quite  promising 
for  on-line  applications.  However,  the  disadvantages  of  this  method 
are  that  it  is  necessary  to  measure  all  the  state  variables  of  the 
system  and  in  addition,  some  differentiation  operations  have  to  be 
performed  on  some  of  the  state  variables. 

As  an  example  illustrating  this  method,  let  the  plant  to  be 
identified  be  governed  by  dynamical  equations  of  the  form 

^+a|f+x  =  K  u(t )  (11) 

dt^ 

where  a  and  K  are  the  unknown  parameters. 

Equation  (ll)  represents  a  system  whose  block  diagram  is  shown 

in  Fig.  2. 


Fig.  2 — Block  diagram  of  example  system 

In  this  example,  it  is  seen  that  the  environment  affects  the 
gain  and  time  constants  of  the  plant.  The  true  values  of  a  and  K 
are  the  solution  of 

5  I  {S**®  +  *-Ku(t)}8*  (12) 
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The  minimization  of  this  expression  yields 


»I(s)2at-KJiu(t>dt  =  -lC&  +  Oiat 


o 

T 


T 


J  g  u(t)dt  -  K  J  u2(t)dt  -  -  J  (4*  +x)  u(t)dt 


(13) 


Notice  that  the  coefficients  of  a  and  K  in  the  two  simultaneous 

linear  Eqs.  (13)  can  be  easily  generated  using  multipliers,  summers, 

.  ,2 

and  integrators  if  x,  g,  ^  and  u(t)  are  available  to  the  identifier. 
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III.  THE  DECISION  PROBLEM 

Cnee  the  plant  parameters  have  been  identified.,  the  next  problem 
facing  the  adaptive  controller  is  the  decision  problem.  A  "control 
function"  or  a  "control  lav"  has  to  be  synthesized  to  minimize  a 
suitably  chosen  performance  index.  Here  a  control  function  is  defined 
as  a  control  signal  which  is  only  a  function  of  time.  A  control  lav 
is  defined  as  a  control  signal  which  is  some  function  of  the  state  of 
the  system.  The  control  lav  may  involve  time  explicitly. 

The  decision  problem  of  adaptive  control  is  equivalent  to  the 
optimal  control  problem  in  automatic  control  theory.  Two  methods  of 
solving  the  decision  problem  will  be  outlined  with  reference  to  a 
fairly  typical  problem  in  optimal  control  theory. 

Let  the  Identified  plant  be  governed  by  dynamical  equations  of 
the  form 

x  =  f(x,u)  (lM 

The  notation  in  Eq.  (l4)  is  the  same  as  in  Eq.  (l),  and  the  comments 
made  in  connection  with  Eq.  (l)  are  also  pertinent  to  Eq.  (lM* 

Let  the  plant  be  in  an  initial  state 

x(o)  =  c  (15) 

Determine  u(t)  to  minimize  a  performance  index  of  the  form 

T 

I(u)  =  J  g(x,u)dt  (16) 

o 

where  g(x,u)  is  a  scalar  valued  function  of  its  arguments. 
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More  generally,  certain  constraints  like  "bounds  on  u(t)  may  "be 
placed  on  the  minimization  problem.  However,  such  constraints  may 
he  easily  eliminated  by  suitable  transformations.  (For  example, 
see  Ref.  11.)  Hence,  only  the  unconstrained  optimization  problem 
mentioned  above  will  be  considered  to  outline  the  methods  for  solving 
the  decision  problem. 

The  Euler  equations  for  the  optimization  problem  are  Eq.  (l^) 
together  with 


i  =  -  s*  -  tl  X 


x  - 


(17) 


where 


gu  +  <  X,  fu  >  =  0 


S. 


r 


n 


(18) 


(19) 


and 


f ' 
x 


u 


*1 

5T 

n 


nr 

•  •  • 

df 

n 

3T 


df 

n 

df 

n 

ST  , 

n  J 


(20) 


(21) 


In  Eqs.  (17)  and  (l8),  X  is  an  n-vector,  the  lagrangian  multiplier 
vector . 


a 
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Equations  (17)  and  (l8)  have  to  be  solved  with  the  boundary 
conditions  given  by  Eq.  (15)  and  the  free  end  condition 

\(T)  =0  (22) 

The  Euler  equations  can  be  solved  by  the  method  of  quasilineari¬ 
zation  to  yield  the  optimal  control  function  or  open  loop  solution 
in  the  form 

n  -  *(t)  (23) 

and  the  optimal  system  trajectory  in  the  form 

x  =  cg(t)  (2h) 

A  solution  in  this  form  is  unsatisfactory  in  general  and  a 
control  law  is  usually  des ired .  It  is  not  easy  to  obtain  the  control 
law  from  the  control  function. 

In  many  practical  situations,  even  if  a  control  law  can  be 
synthesized,  it  will  not  be  a  satisfactory  solution  because  of  the 
complexity  of  the  dependence  of  the  control  law  on  the  state  of  the 
system. 

Often,  the  form  of  dependence  of  the  control  law  on  the  state  is 
known  beforehand  except  for  a  finite  set  of  parameters.  The  known 
form  depends  on  the  manipulations  that  are  possible  with  the  available 
physical  equipment. 

The  control  of  a  plant  with  a  controller  whose  form  is  specified 
beforehand  will  be  defined  as  specific  control  and  the  problem  of 
choosing  the  parameters  in  the  specific  controller  to  extremize  a  per¬ 
formance  index  will  be  defined  as  the  optimal  specific  control  problem. 


It  will  be  assumed  here  that  the  decision  problem  of  adaptive 
control  systems  is  equivalent  to  the  optimal  specific  control  problem. 

The  optimal  specific  control  problem  will  be  stated  in  the  fol¬ 
lowing  manner. 

Given  a  plant  with  dynamic  Eq.  (l4)  and  initial  condition  (15) 
determine  the  unknown  parameters  in  a  control  law  of  the  form 

u  =  A(£,b) 

where  y  is  a  p- dimensional  subset  of  x  and  b  is  a  q-dimensional 
parameter  vector  such  that  a  performance  index  of  the  form  (l6)  is 
minimized. 

The  following  methods  are  proposed  for  solving  the  optimization 
problem. 

A.  THE  QUASimtEARIZATION  METHOD  FOR  SPECIFIC  CONTROL 

The  optimal  specific  control  problem  may  be  rewritten  as : 

Given 

x  =  h(x,b)  (2^) 


with  x(o )  =  c  where 


h(x,b)  =  f(x,*(£,b))  , 


determine  the  parameter  vector  b  such  that 

T 

j(b)  =  J  k(x,b)dt 

o 


(27) 


(28) 


is  minimized. 
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In  Eq.  (28) 


k(x,b)  = 

g(x,£(£,b)) 

(29) 

Ey  adjoining  the  equation 

• 

b  = 

0 

(30) 

to  Eq.  (26)  the  specific  control  problem  is  reduced  to  an  ordinary 

problem  of  minimization  of  an  integral  with  differential  constraints . 

The  Euler  equations  for  the  minimization  problem  are,  in  addition  to 

Eqs.  (26)  and  (30) 

• 

X  = 

1 

IX* 

1 

IX* 

(31) 

• 

n  = 

~  ~  **b  ^ 

(32) 

vhere  X  and  n  are  the  Lagrange  multipliers  associated  with  Eqs.  (26) 
and  (30)  respectively.  The  quantities  k^,  k^,  h^  and  h^  are  defined 
in  a  manner  analogous  to  Eqs.  (19)  and  (20). 

The  natural  "boundary  conditions  for  the  problem  are 


X(T)  =  0 

(33) 

n(o)  =  p(T)  =  0 

(34) 

The  Euler  equations  with  the  boundary  conditions  may  now  be 
solved  by  the  method  of  quasilinearization  to  yield  the  values  of  the 
parameters  in  the  optimal  specific  controller. 
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B.  OPEN  LOOP  -  CLOSED  LOOP  METHOD  FOR  DECISION 

Philosophically,  this  method  is  different  from  the  previous 
method.  The  solution  here  depends  on  the  solutions  given  by  Eqs.  (23) 
and  (24)  vhich  do  not  depend  on  the  form  of  the  specific  controller. 

In  general ,  the  solution  using  dais  method  will  result  in  a  slight 
degradation  of  performance  compared  to  the  previous  method.  However, 
this  method  will  yield  solutions  vhich  will  be  easier  to  instrument. 
The  computations  necessary  with  this  method  are  often  easier  to 
perform. 

Recalling  that  Eq.  (24)  represents  the  optimal  trajectory  without 
the  specific  controller  constraint  and  Eq.  (26)  represents  the  system 
equation  with  a  specific  controller,  it  is  easy  to  see  that  if  there 
exists  a  set  b  such  that 

<p  =  h(<p,b)  (35) 

then  the  set  is  the  optimal  parameter  values  for  the  specific  con¬ 
troller.  However,  in  general,  Eq.  (35)  cannot  be  satisfied. 

Equation  (35)  may  be  rewritten  as 


9  -  h(sp,b)  =  0 


(36) 


Since  Eq.  (36)  cannot  be  satisfied  in  general,  one  intuitively 
feels  that  an  acceptable  solution  may  be  one  which  makes  the  left- 
hand  side  of  (36)  "close  to  zero,"  the  closeness  being  defined  in  a 
suitable  manner.  In  this  Memorandum,  the  set  b  will  be  picked  such 
that 


x 
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T 

J  <  9  -  ,  cp  -  h(cp,b)  >  dt  (37) 

o 

is  minimized  with  respect  to  b. 

The  method  of  solving  this  minimization  problem  is  straight¬ 
forward  and  is  exactly  similar  to  the  method  used  in  the  differential- 
approximation  method  for  identification.  The  partial  derivatives  of 
Eq.  (37)  are  taken  with  respect  to  the  components  of  b  and  equated 
to  zero.  This  will  yield  q  simultaneous  equations  involving  the  q 
component  of  b.  The  solution  of  this  set  of  equations  yields  the  op¬ 
timal  specific  controller. 

(12) 

A  useful  alternative  method  is  proposed  by  Strejc.  ' 
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IV.  DISCUSSION 

Cnee  "the  controller  parameters  axe  determined,  the  modification 
problem  consists  of  resetting  the  parameters  to  the  determined  values. 
This  is  the  easiest  part  of  the  over-all  problem  of  designing  an 
adaptive  controller. 

Unified  methods  for  solving  the  identification  and  decision 
problems  of  adaptive  control  are  proposed  in  this  Memorandum.  Use 
of  these  methods  will  permit  the  same  equipment  to  perform  both 
operations.  This  will  result  in  cheaper  cost  for  adaptive  controllers. 

To  utilize  the  methods  here,  the  adaptive  controller  can  be 
either  digital,  analog  or  a  combination  of  both. 
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